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12 ALSTRACT 


The deveiopment of resource sharing networks can facilictate the 
pruvision ot a wide range of economic and rel.able computer 
services. Computer Communication Networks allow the sharing of 
specialized computer resources such as Gata bases, programs and 
hardware. Such a network consists of both the computer resou:ces 
and a communications system interconnecting them and allowing 
their full utilization to be achieved. In addition, a resource 
sharing network provides the means whereby increased cooperaticn 
and interaction can be achiev2d between individuals. This pape. 
provides an introduction to computer networks and resource sharing 
and discusses some aspects of distributed computation. 
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I. INTRODUCTION 


The interaction between computers and communications has 
Steadily developed over the tlast two decades, While many 
universities, govermment agencies and business firms prefer to 
make use of cheir own computers, an increaSing number of people 
are uSing communication facilities to access commercial computer 
services {l]. Time~sharing and batch processing services are 
offered in most major U.S. cities or are accessible via 
telephone circuits, and communication charqes for local telephone 
access to these services are, in general, substantiallv lower 
than the computer. charges. As the use of comnuter services 
increases, the demand for reliable and low cost means of 


communicating over wide geoqrapvhic areas aiso increases. 


For many years, networks of interconnected computers have 
been planned or under studyv, and more recently several have been 
under development [16,18,24]. A common objective underlying the 
interest in these networks has been to demonstrate that economic 
savings or increased capabilities are possible bv sharing 
computer or communication resources. Proaram access to 
specialized data bases 1S an important examnle of resource 


sharing in a computer to comnuter network. 


The growing usage of these data processing services and the 
objective of sharing resources raises communication issues far 
more extensive than those of achieving increased canrahility and 


lower costs in the telephone network, or developing improved 


Ea aes 
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communication services [2]. They involve a number of complex 
requlatory issues, the need for common methods of access to and 
interchange between data processing systems, the pooling of 
compute: resources for increased utilization and reliahility, the 
provision of specialized services, data conferencing and_ so 
forth. A set of associated regulatory issues involving 
telecommunications policy have been raised and are under 
intensive study. Are separate common carrier data networks 
desireable or not? What is the most effective wav to plan for 
interconnection of data networks and how should their usage be 


tariffed? 


It is too early to accurately predict in what way this 
interaction of computers and communications is likely to evolve, 
The technology is changing rapidly, and regulatory policies are 
in flux. If communication costs are not to dominate the overall 
cost of uSingqg remote data processing services, technoloaical 
advances must allow communications at substantially lower per bit 
costs than are possible with the current switched telephone 


plant. 


In this paper, we present one viow of computer 
communications network development and explore a numner of the 
important issves in distributed computation which h&ve arisen. 
Th 3 ‘iaper is neither a commleteiy generai treatrent of computer 
networks, nor a full case study, but rather it contains sclected 


aspects of the two. the reader will, no doult he alle to 
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identify where general considerations give way tO specific ones 
derived primarily from the author's experience with the 
development of the ARPANET* (7,18). It is impossible for this to 
be an exhaustive treatise, or even a coinprehensive one, and no 


such attempt is made, 


*The Advanced Research Project Agency Comrucer Network. 
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II. Distributed or Concentrated Resources 


Many economic factors support the conclusion that geographic 
"clustering" of computers is a desirable strategy for computer 
service organizations [9]. One possible adcvantane is better 
equipment utilization due to the poolinq of resources, 
Clustering implies that a single maintenance staff (which is 
often underutilized) and scarce system personnel can support more 
equipment, more reliably, and that space, auxiliary equipment and 
overhead can be consolidated. In fact, several cormercial 
time-sharing firms have already chosen to concentrate their 
computer resources in a= small number of geogravhic areas. In 
contrast, however, many individual research or development 
machines under private or government ownership are distributed 
throughout tne U.S. The valuable resources on many cf these 
machines provide a strong incentive for them to be made available 


to users and comnuters at many cther locations [1,11,33]. 


The location of computers at a few geogranhic locations 
requires chat both local and remote users be provided with an 
economic and reliable way to acces; the’ service. The switched 
te,ephone network currentiy appears to be a poor candidate to 
provide the long distance communications service. In addition to 
being considerably more costly than local serv.ce, the error 
performance on long distance circuits 1s degraded from shorter 
circuits and is insufficient for many computer applications. In 


addition, freauent disconnections, busy sicnais, etc. during 


i 
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peak traffic hours often make its usaae irconvenient. ‘These 
factors, coupled with user desires for increased bandwidth, lower 
set-up times, and more suitable tariffs have encouraged several 


vendors to competitively enter the conmon carrier market [23]. 


The tariffs and the technical characteristics of the circuit 
switched telephone network reflect the nature of voice 
communication recuirements that are quite dicferent from tnose of 
computer communications. Due to the bursty nature of comnuter 
traffic and the extremely low utilization of a tvnical voice 
grade circuit by a terminal, a substantial portion of che data 
communications capacity in a circuit Switched system is simply 
not used. This results in inefficient utilization of telephone 
company resources from tite user:. point of view. Frecuency or 
time division multiplexing techniques have been usefully applied 
for deriving individual chanrels, but the statistical nature of 
computer traffic makes fixed ailocation strategies such as these 


inefficient or unacceptable. 


On the other hand, statistical multiplexing techrioues allow 
these circuit resources to be more widely shared , at the 
possible expense of occasional delays in transmission, Messaqde 
switching emplovs a generalized yorm of multiplexing for a 
network environment that allows all circuits to be shared among 


all users in a Statistical fashicn without being allocated in 


advance, 
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This has been the motivation for the development of new 
communication systems as well as combined computer communication 
networks. The construction of both common carrier data 
communication systems and "private" networks (using leased ccmmon 
Carrier faciiities) is a natural outcome of the need for economic 
and izeliable communication between users and geographically 
distributed computers. In addition to rotential cost savings, 
many of these networks provide error control, as well as 
asynchronous operation, local echcing, speed and code conversion, 
which are better suited to data communication with computers than 
use of the telephone network alone. e# reevaluation of the tariff 
Structures for dat. communication has recently been undertaken by 
the F.C.C. and efforts are being made to provide the public with 
data communications service having lower error rates, smaller 


service charging intervals, and faster set-up times than _ the 


switched voice network currently provides. 
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YII. Computer to Computer Communications 


A computer network is a complex collection of many tynes of 
resources, including data bases, proarams, onerating systems, and 
Special purpose hardware, all of which are capable of being 
accessed from any other resource in the net. Computer tc 
computer communication is necessary to achieve effective resource 
sharing, but the ability to transfer information between machines 
does not autematically result in useful machine to machine 
interactions, Aroused by the exciting possibilities in using 
multiple machines, system designers have recently begun to 
provide the major technicai effort required to achieve effective 
computer to computer communication. The existence of the ARPANET 
is having precisely this effect, and as a result the extent of 
computer to computer interactions is certain to grow 


supstantially in the next few years [6]. 


The ARPANET is one of the most advanced examples of a 
computer communication network [8,16,18]. It consists of a 
geographically distributed set of different computers, 
interconnected by a communication system based unon very fast 
response (interactive) message switching. This network was 
developed to ultimately allow economic and reliable sharing of 
specialized computer resources. ‘The ARPANET has demonstrated the 
feasibility of messaae switching technology, illustrated its 
advantages, and fostered the development of techniques for 


computer to commuter communication. It is interesting to note 


Report No. 2459 Bolt Beranek and Newman Inc. 


that the ARPANET was originally eesigned with the notion of 
computer to computer communication in mind. It has subsequently 
been extended in capability to allow users wich terminal 
equipment but no computer to connect to the net and communicate 
with computers and other users. In this sense, tie ARPANIT has 
taken the opposite approach from every other network Gesigned 


with user access Oriqinally in mind. 


For many years, the National Physical Laboratory (NPL) in 
England has experimented with the use of “single packet" messaues 
for switching in the "local area of a data communication network" 
(ale A number of — terminal devices were successfull; 
interconnected into a Local network at NPL and recently they have 
been concerned with extending the local network into a 
distributed network [3C]. A computer to computer network is also 
under development in France to allow data sharing without costly 
duplica’i. . of files, and its attendant problems of control, 
updating, security etc. Certral files, each accessible via a 
locai computer, will be made accessible to other computers and 
hence to an extended uSer community. This network is expected to 
use a message switching technigue similar to that used in by tne 
ARPANET in the U.S. in addition, networks are under desiadn or 


development in other countries (e.q. Canada, Japan). 


In general, the properties and structure of a computer 
communications network must reflect the overall requirements for 


which it was desiaqnec. This mav consist of high speed 
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(meqabit,’second) eClrcuLts for Yapid commuter toe computer 
communicaticn, or lew Sneed (voice and telearaph qrade) circuits 
for terminal access or slow speed = cormnunications; it may Le 
Circuit switched or message switched, etc. Whatever its detailed 
Structure the network contains a communication svster (private o. 
common carrier) and a set of computer svstem resources and users 
that interact via the communications svstem., This svstem is also 
called a communication subret or simolyv a subnet for short. This 
organization not only characterizes the ornanization of 
Geouraphicailv distributed networks, but can also serve as a 
model for the local structure of a single cornuter conplex [3]. 


Its structure is therefore quite fundamental. 


In operating a computer communications network as a 
"marketplace FOr commucer related services, a number. of 
important issues arise [34]. We allude to a few of them ore. 
What criteria are appropriate to deterrine whether a service may 
be removed from the svstem? When and where shouid additional 
services be incorporated anc what procedures are needed to 
maintain effective comnetition? What Subnet changes are 
appropriate for chances in the distribution of resources? The 
total operational procecure should also include a_estrategqy for 
utilization of the resources consistent with ifs irtende:! 


functions (e.q. load Sharing, data sharing etc.) 


An overridina concern of the network desidqn is the overal] 


reliabilitv of the comrunications and computer resources. For a 
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user to entrust his computing to a network, he must develon 
confid2znce in its availability when he needs it. It must be 
convenient to uSe and it must provide a helievable guarantee to 
maintain standard and expected grades of service. An investment 
in time and energy to use a network resource can be negated bv 
the failure to maintain a consistent service offering. Insuring 
that proper concern e:.j)sts for the remote user of a comouter 
resource is an important administrative problem that affects 


almost every phase of computer network development. 


Sig= 
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IV. MESSAGE SWITCHED COMMUNICATIONS 


Since the messaqe switching technology is not as well 
established as the circuit switched technology, the fundamentals 


~ 


of its operation are reviewed in this section. Considerahle 


Xr - 


G@iscussion on the nature of these two switching doctrines is 
taking place. Are they merely aierorene ends of a common 
Spectrum (with a kev variable such as packet’ size), or are they 
fundamentally different communication techniques? An araument in 
favor of their similaritv is that both tvnes rely on store and 
forwarding of data, whether a Single bit 1S transiently stored, a 
byte sized envelone or a larger sized packet. The most 
Significant external characteristics that "annear" to distinguish 
the two svstems are tnat 1) circuit switching svstems are hetter 
equipped to mairtain a tine frane for users tnat require 
ccatinuity in transmission, as in speech, while 2) messaae 
switching systems allow sneeu anc code cenversion , thus 
nerm tting direct connection of and communication hetween devices 
of widely varying tyre. But it 1S possikfle to mask even these 
"seemingly" essential differences by the provision of a smali 
amcunt of bufferina and "byte manipulation" capabliitv at the 
periphery of eitner system, It 1s actually tne manner in whicn 
internal system resources are mManaavl and utilized tnat provices 


a useful measure of comrarison hetween then, 


Brieflv, in a cirenit Switched network, the source an? 


destination are coniuected by a dedicated communication path that 


Se 
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is established at the beginning of the connoaction and broken at 


the end. This tvoe of connection was specifically selected for 


us2 in switched telephony, where subscribere require a continuity 


ix. voice transmission and reception. Since the communication 


path remains fixed for the duration of a ccnversation, the output 
speech signal arpears to be a time translate of the input speech 
Signal as far as the ear can tell. In addition, for most voice 


conversations, the allocated analog voice channel is used ina 


fashion that seems reasonably efficient to the averaqde user. 


To establish a connection, the subscriber provides the local 
central office with an address which is used in setting ur a 
path. Central effice equinment detects offehook, provides a 
Gial-tone, retains dialea digits, generates ringing, busy signals 


etc. In the current telepnone piant, long haul circuits are 
primarily multiple») d anaiog channels. Routing selection is 
performed using a set of prespecified paths and usually based on 
the first few dialed digits. Call set-up times generally take 
4hetween 5 and 25 seconds depending upon the number and tyne of 
central offices in the link and the amount of traffic. Recent 


experience has also indicated that reliability and overload 


problems are becoming increasingly prevalent in certain hiqh 


density population areas. 


A message switching system accepts, transmits, and delivers 
discrete entities called messages. In such a svstem, no physical 


path is set up between the source and the destination and no 
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resources (€.q. Capacity, buifer storaqe etc.) are allocated to 
its transmission in advance. Rather, the source includes a 
destination address at the beainninq of each messaae. The 
message Switching system then uses this address to gquide the 
message through the network to its destination, provides error 


control, and notifies the sender of its receipt [14s]. 


A simple form of messaqe switching svstem emnloving a single 
central switchinq computer is commonly referred to as a “star" 
confiquration and has all its lines connected to the central 
message switch. For many local anplications this configuration 
can be quite practical. Three of its main disadwantagqes are 1) 
the central switch may be an unreliable jink which will disrunt 
all communiczations if it fails, 2) the total circuit mileadqe for 
geographically distribvutec users to connect to the switch mav be 
substantially laraer tiian necessarv, resultino in excessive 
communications cost, andl 3) every circuit failure can result in 


some loss of uSer communications. 


A distributec messaac switching svstem 1S one in which many 
distributed switzchina comnuters are employed and the network 
control is decentralized in such a wav that the failure of any 
switching computer disrupts communications only for its local 
customer. The distributed svstem is usuelly more economic” and 


reliable than aie star configuration for handling qeoaraphicallyv 


distributed users. 


i 
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The components of a messaqe Switching Svstem are dedicated 
point to point communication circuits and switching nodes which 
innerconnect the circuits in Sucn a wav that a messaqe arriving 
on one circuit may be transmitted out another. Communication 
over a message switched svstem occurs via a sequence OF 
transmit.ted messages , each consisting of its address followed by 
text. The address is inspected by each node in routing the 
message to the next node on the way to its destination. In the 
ARPANET, one or more computers may be directly connectea to a 


node and are known as Host computers, or Hosts for short. Tue 


nodes are called Interface Messaae Processors or IPs for short. 


A distributed messaqe switched network, Such as the ARPA’! T, 
contains no mass storage, and as little buffering in the nodes as 
necessary to utilize the full carnacity of the communication 
circuits*. The network desiqn allows a messaqe to remain in the 
net only as long as necessary to transport i1t = froi fourcc to 
destination; no long term storaqe is providea in the 
communication svstem. Messaaes that cannot be delivered to the 
destination are simnlv not accepted into the net and must be 
retransmitted at a later time. Clearly, one or more Hosts on the 
net with low cost per bit bulk storaqe could provide or even be 
dedicated to providing long term storage of messases with 
Subsequent automatic retransmission. 

* The Defense Departments Autouin networt., however, emrlovs mass 


Storage in the communicaecven networ; for defcrrea retricval and 


delivery of messaaes. 


ee 
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The combinatorial asnect of the interccrnection of larqez 
numbers of computers is) an important consideration in networrx 
design. Each computer in a message swit:ching system is connected 
to the net via a Single full duplex cnannel to its I! over which 
messages are multivlexed. Tnis Single connection to the networl 
makes the computina service accessible to all computers and all 
users on the net. Furthermore, all users and ail computers) on 
other digital networks can access this computer bv the simnle 
expedient of a Single interconnection between nets. Thus, not 
only is complete digital access possible, it is achieveau ina 
Strikingly economic wav for each installation, Tnis  techniaue 
solves a massive combinatorial access nroblem with a single 


economic stroke, 


In Figure 1, we show the communications portion of the 
ARPANL’Y as of April, ly/2 when it consisted of twentv-four nodes 
and twenty-eight circuits. Since that time it has grown to over 
thirty nodes. Each node is a possible source and destination of 
messaaes. We assure (for the moment) that messaces nav he of 
variable length una to a raximur of ivcs bits* and are known as 
packets while ir the necwor*. The path taken bv ai packet 
traversing the net from node 12 (UCLA) to noee 6 (“I'T) is 
indicated by arrows in the fiqure. The circles irdicate tne 
nodal processors and the lines indicate svncnronous noint to 
POLNne ‘circuits: 


*In the ARPANI’T, messaacs mav actually vary ur to 46.95 hits in 


length. 
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The message enters the net at node 1, which examines its 
address and decides to transmit it out “ts circuit to node 7 
(RAND). Upon receipt, node 7 examines the address and decides to 
send it to node 21 (TINKER), which in turn sends it to node 5 
(BBN) which sends it to node 6 (NIT). Node 6, discovering tne 
message is for itself, replaces the destination address by tie 
source address (which is carried along by the message switching 
systen) and "delivers" the message. The text of the messaade thus 
appears at the destination exactly as iv was transmitted and the 
address portion identifies the sender. After delivery, the 
sender is notified of its receipt by a small messade that goes 


yack across tne network. 


An important part of a computer network desian is the 
specification of the location and capacity of all circuits in the 
fet. Preity kilebit/second circuits are currently used in the 
ARPANET to achieve an average delay of 5.2 seconds or less. 
Programs have been developed that jterative.y analyse various 
possible network confiaurations and select reliable, high 
throughput, low cost desians throuah the use of circuit exchange 
heuristics {20,22]. Analytical techniques have been developed 
for estimating the averade transit time delay under assumed 
traffic loads. These techniques show that the delav remains 
almost that of an unloaded net until the capacity of one or more 


"cutsets” beqins to saturate ["]. 


An important design consideration is the method for 
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dynamically selecting routes. (We assume that routes are not 
allocated in advance.) A central controller could provide the 
routing information and distribute it to all the processors, or 
the processors could collaborate in computing the routing 
information directly. This is but one of many instances of a 
design choice between distributed and centralized control. In 
the initial ARPANrT design, the route selection is performed 
independently by each IMP according to a distributed routing 
algorithm. Routing information is stored in a table and 
individually maintained by each IMP for rapiu_ look-up. det: 
identifies the output line to select for each destination and is 
updated accordina to a rule evaluated periodically (e.q. every 
half Second) .« It could alsc be evaluated asynchronously 
(whenever status changes occur) or a combination of both. In the 
Simple algorithm used in Fiaure 1, 2acn IMP senis the message on 
its choice of a path with the fewest intermediate IMPs anu, usina 
the update procedure, each IMP adapts its routing to other IMP 


and circuit failures. 


A Simple method for imolenenting this algorithm is for each 
IMP to kevn a tahle with the count of the num-er of I'ps on the 
shortest path to each destination which it frequently transmits 
to its immediate neighbors. Each IMP also announces to its 
neiahbors that it is J IPs awav from itself. Upo: Yreceint of 
the routinq information from its neiqhbors, the IMP increments 


the neighbors counts and keens the lowest value for each 


destination. 


Sie 
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Each IMP buffers a packet until receipt is acknowledged by 
the adjacent IMP. A cyclic checksum, generated in hardware by 
each IMP, is appended to the transmitted packet for error 
control. If an error is detected by the hardware at the adjacent 
IMP , or no buffer space exists, the packet is simply discarded 
and will shortly be retransmitted by the neighboring IMP when a 
condition (such as a time-out) occurs and no acknowledgement is 


received, 


The design of an efficient network wihout mass_ storage 
requires that the number of buffers be kept to a minimum, and 
that they be used so that each IMP is able to use its circuits 
efficiently and to their maximum capacity. This means that the 
minimum number of buffers must be at least as large as the number 
of full sized packets which must be stored from the tine one full 
Size packet is transmitted until its acknowledqment returns, 
This number is determined by the circuit propacation delav, the 
packet size and the circuit data rates, as well as the total 
number of circuits. To utilize these buffers efficiently, stored 
packets must be quickly released upon receipt Of their 
acknowledgement or activated for retransmission, aS appropriate, 


in a timely way. 


Each IMP contains only a small amount of buffering for 
messages in transit and no mass_- storage, anc a fiow control 
Strateaqy is needed to insure that the IMPs do not _ become 


"congested" thus preventing useful data from being comnunicated. 


= leo 
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any event. For multi-access satellite circuits, however, the 
extra traffic during lictt loads mav interfere with other 


processors sharing the same channel. 


In an unloaded net, the transit time is determined primairily 
by the number of IMPs encountered in the routing and the time for 
the packet to pass from one IMP to the next. This, in turn, is 
determined directly from the packet length, the circuit data 
rate, and the speed of light propagation delay. Under increasing 
traffic loads, the transit time also begins to increase due to 
occasional delavs in the I!Pps, However, i1f the network is 
desiqned to begin rejectinag the further input of traffic as the 
Cavacity limitation of the network is aporoached, these delavs 
can be kept to a few tines that of an unloaded net. Traffic is 
thus queued outsice the net (rather than allowed to enter and be 
queued inside the net) so the nominal transit time duriny peak 
traffic is not verv different from that experienced in an 
unloaded net. In these cases, an attempt must be made to insure 
that the effective bandwidth is shared "fairly" amona all_ the 


competing sites. 


Network uSage generally requires a@ combinavion of short 
transit times for interactive usage and high bandwidth for vile 
transmission. These two objectives may be attained with single 
packet messages. To achieve interactive transit times, no set-up 
delay must be incurred. A simple way to achieve this is for the 


source IMP to retain a copy of each packet which is nominally 


wh 
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discarded after the delivery is made, but retransmitted when, for 
lack of buffer sSnace, the original is discarced at the 
destination IMP. To achieve high bandwicth, enough messages must 
be allowed to enter the net between source and destination so as 
to £111 the "pineline", but this flow must be akle to be readily 
quenched at the source when the buffer space at the destination 


IMP begins to fill. 


The current ARPANET desiaqn actually allows variahle length 
messages with a maximum size just over 8°(f' bits. The messane is 
partitioned by the source IMP into separate 1/'[%f' bit packets to 
speed its transmission throudqh the network. Each nacket mates 
its wav to tue destination independently where it may conceivably 
arrive out of order. These packets could be reassembled into tne 
proper order bv the destination Hest (using sequence numbers), 
but when the assumption 1s made that the communications net 
should preserve seauencing at least at the level of a= single 
process to process conversation, the IMPs are obliged to 
reassemble the packets. The destination Host thus receives’ the 
text of each messadqe exactly as it was transmitted in a single 


block transfer. 


When these laraer messaqes are used and the IMPS undertake 
the responsibilitv for reassembly, yet another type of congestion 
phenomenon called reassembly lockup is introduced [13]. The flow 
control mechanism which is used to prevent the backun of messaqes 


in the net is also powerful enoudh to prevent the lockup problem. 
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But in its application, it ian subject long messages to set-up 


delays and thus delay succeeding short messases from its Host. 


If “sufficient" buffer space were available fer reassembly 
at the destination IMP there would be no "a priori" compelling 
reason for the subnet to prewlude a Host from sending full 860 
bit messages (or even somewhat larger ones). However, the 
presence of 8)J0{) bit messaqes mav neciceably delay shorter 
messges from other Hosts 1) while it is being delivered to the 
destination and 2) by tying up eight buffers rather than one 
during reassembly. This provides one valid reason to restrict 
the Hosts to sinale packet wessaqes, if these delays beccme 
Significant. However, a& we indicate in section V, there mav be 


other factors which favor rete.tion of the laraer size, 


If there is a fundsrmental distinction hetseen circuit 
switching and messaqe switching, it is undoubcedly in the way 
internal resources dare managed. For example, circuit switching 
requires that network bandwidth as well as local control 
equipment and storage be allocated to a qiven transmission 
circuit in advance, whereas a message Switcning systen does not. 
Secondly, the presence of cirsuit switched routes means that user 
messages are identified by their circuit and no user control 
Signalling need accompany the transfer of infec “ation. In 
message switching, however, all record of activi‘y (except 
accounting) associated with a messaye 1S contained in the 


message, which vanishes when the messaae leaves the system. This 
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Siqnalling information, in the form of an address, must accompany 
each message ani the message must be examined and processed at 


each stage of the transmission process. 


Two practical ccnseauences of the difference are that the 
ehrcuLt switchec svstem usually recuires a non-trivial set-up 
time to allocate resources. Messaqe switched systems can avoid 
eet-up delays, but may introduce occassional variations in 
transit tine. These de’ays can generallv be maintained to within 
a few times the delav of an unloaded net, but witier variations 
may result fro.. queueing delays outside t. net, particularly 
under heavy traffic load. Under similar conditions, thouch, a 
circuit switched user micnt fail to obtain a circuit and would 
iicur this probabilistic Situation on  subseauent tries. Any 
allocatec but idle channels are sirmly unavailable at this tine 


to handle these overliad conditionc. 
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V. NETWORK USF OF INDIVIDUAL COMPUTER SYSTEMS 


The term network has been used and misused in a variety of 
wayS. Some people have referred to the use of dial-un facilities 
to access a Single computer as a network. Others have referred 
to any interactin:. between computers and a communication system 
as a network capability. Several distributed networks were 
developed to allow simple forms of communication between 
identical machines using Standard dial-up or leased voice 
circuits, thus providing a convenient way to transfer jobs and 
files and to maintain and update the systems in the net. This 
latter application exemplifies a true networking activity, even 
though it only concentrates on selected aspects of computer 


resource sharing [19,21]. 


In general, only a subset of the network sites possess 
computing power, and certain of ther will offer reqular service 
to users via the network. Other sites may choose to offer 
service only on a limited basis, or to cooperatively interact via 
the net with selected co-workers, but not offer general service. 
This latter Situation 1s more likely to occur for many 
Specialized research facilities. In addition, larac private 
computer centers as well as commercial firms may welcome the 
opportunity to connect their systems since it offers a larqe 


votential: market for usage of unused capacity. 


Host service on the network ought to ke as reliable as the 


communications, although this objective is often difficult to 
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achieve. For example, in tne ARPANET, total uptime of the IMP ac 
any site is currently on the order of 93 or 99%, while Host 
availability is generally no hiqher than 908%. It is certainly 
possible to improve on this score; some commercial firms claim to 
provide over 99% availability of service, and certain private and 
government systems must obviously be operated with near perfect 
reliability. The airline systems and the computers in the space 


program provide two key examples. 


ae Networ’ Access 


In a network that supports computer to computer 
communication, user grouns with a local comnuter can access 
another computer in the net bv first loaqacing into their home 
computer and then into the other comnuter, using the home 
computer in a transparent moce as a switch. Howeve::, this is) an 
expensive way to access another computer since it incurs charaes 
in two computers and ties un jobs in hoth. Furtnernmore, since 
Yosts may he unreliable, the connection 1s more vulnerable than a 


-trect connection into the other machine. 


Sites with exnensive commuter installatiors midqht deen it 
economic to depart from their maciiine if “equivalent" service can 
be economically obtainee via the net. In general, such ae site 
requires the ability to service local users with a $fvll 


complenent of terminals and peripheral devices such as teletvnes, 
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graphics displavs , line printers, magnetic tapes, and possibly 
other mass storace. In addition, many sites with no computer 
will derive maxinum benefit in connecting to a net if the full 
range of peripherals can be provided locally. Other users, 
however, may be satisi'scc with a simrler approaci that proviidcs 
terminal access to remote computers but does not handle 


peripheral devices other than line printers, 


For this latter class of users, an addition mav be mace to 
the IMP which allows a set of terminals to be directly cennected, 
This addition consists of multiplexing edquinnent that collects 
characters from the terminals and pechadss them in the form 
Suitabie for deliverv to the IMP. Likewise, it receives messaacs 
ErOM: the ei ane (SOr es ect SoCita race b rs: | Out ba tence dels 
terminals. This addition rennires haraware and sottwaure desianecd 


to make tne “set of “terminals inter sace to tre network as a 


"“mini-host", and this IMP is referec’ to as a Terminal IP, 


A more elaborate anproach is annrepriate for user sitas that 
wish to supynort manv different types of neripheral enuinnent. 
Since their charact@ristics and oneration can vary witelv, device 
dependent proqraminag 1s mancatoryv, and substantial bof*orindg mav 
be required for the nidqher Sneenw devices. Furtherrors, sites 
with mess storage will generally wish it to be accessible fram 
other ‘compiitters an thie “net, whict gererally “rec.ixves the 
implementation of a full set of standard and snecialized natwork 


protocols. These considerations make it appropriate to provice a 


ni 
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separate processor devoted to the handling of peripherals. 


This latter anproach 1s particularly desirable for a site 
which is both a user site and a Server site. Tne architecture of 
such a site should be orqanized so that if the servina site cpu 
is down, local users can access other network. comnuters and local 
mass storage can be be accessed by them over the net. Similarly, 
if the local storage should fail, others across the net can 
temporarily replace them. If the net fails, local users’ can 
Still obtain full access to the local svsten, Only if both the 
local cpu and the network fail will the users be unable to obtain 


computation, 


Modularity ant logical reconficeuration are conveniently 
achievable in this wav. Substantial progress in the design of 
moagular communication oriented arcnitectures can be expected from 


innovative usage of interconnection ideas [3,4,5]. 


b. User Requirenents 


Let us now turn our attention to the use of these facilities 
by “he user. We note three potential locations where user 
computation can be performed = in the termine] itself, ain the 
peripheral processor (or Terminal IP) “or “an “the Host 
computer(s). Although the bulk of the “pure computation" will 
undoubtedly take place in the Host computers, some asnects of the 


processing must, in general, be distributed. For example, local 


227 


(See ee 


Report No. 2459 Bolt Beranek and Newman Inc. 


echoing iS required to obviate the otherwise noticeable effects 
of speed of light prepaqation delav as on satellite links. This 
raises the important question of location of functions ina 
distributed network. In other words, what intellinqence is needed 


to allow distributed system usage and where should it be placed? 


Let us concentrate on the echoing problem for the moment. 
As a general rule, a remcute use. should see the same output and 
otherwise obtain service from a remote Host as if he were a local 
user, Yo achieve this objective, the users local svstem 
(programmable terminal, “mini-host", etc.) must have considerable 
information available to it about each subsvstem in use at the 
remote Host. For example, a sinole local echo/reamote echo 
strategy is generally insufficient to handle echoing for users on 
half duplex terminals, or users on full duplex terminals that 
prefer to type ahead. A remote user editing tne character string 
ABCDEFG can delete the last three chrracters by typing successive 
delete characters (echoea as ) and he sees the output 
ABCDEFG GF E. USina the svstem from a remote site with local 


echoing and typeahead he would see ABCDEFG Ghai. 


More strikina, perhaps, is the remote use of a debugaing 
program DDT. fo examine successive reqisters i2.s - 123 a local 
user would first tvpe 12!:/ to print the contents of the first 
register and then strike successive line feed cnaracters to 
examine the succesSive reqisters. The system woulda respond with 


(say the contents are all zeroes): 


Ne 
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120/ 0 
121/ 0 
122/ { 
123 6 


and leave the cursor following the last 0. A remote user with 


typeahead and local echo would see one of several posible 
responses (depending on the remote systens response to a received 
line feed). Assuming the remote system echoes only the formatted 
data and the local system echoes linefeed as linefeed (no 


carriage return), the outvout would look as follows: 


120/ 


O18 W227 sles 7 
In general, the local echoing svsten should have full knowledaqe 
of the time varvine svntactical operation of the subsystem in 
use. This requires feedback of information about suosysten break 


and separator Characters, control Siqnalling, special] 


conventions, etc. 


Each terminal has characteristics peculiar to it ani a 
Convention “1S <reqiirea for a COmmDUuter “System to “1nitially 
recognize a terminal. Although the remote computer could then 
convert to the characteristics of the terminal, it is far more 
manageable if each terminal could annear to the network as a 
standard terminal emnlovina an aqreed upon set. of characters and 
signalling conventions. One such standare (developed for the 
ARPAIES) 1s 7 bit) USPSCID with the elaqhith bit set to 0. Th that 
scneme, the other 128 possible characters are reserved for 


special control characters, In addition, most terminals need 
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attention to details such aS Carriage return, kevhoard locking 
and unlocking, interrupt signalling, and other peripherals 
indicate out of paper, buffer full, and may require complete 


two-way channel control, etc. 


Local computation is therefore needed for the users terminal 
to interact properly with other remote systems and their 
subsystems. At a minimum, his local computation must allow the 
user to 1) identifv his terminal to the network, 2) Select a 
destination Host, 3) select a transmission mode, 4) perform 
echoing and code conv2rsion, and 5) allow the remote Host to be 


interrupted. 


It seems probable that, in the long run, manv terminals will 
contain mini-processors and thus user proarammina in a separate 
"mini-Host" will be unnecessarv. However, until this nossibility 
is a widespread reality rather than an expectation, users mav be 
hindered if they are unable to provide local user code in_ one 


place or the other. 


c. Message Processing 


Before considering various examples of usage of a computer 
network, let us briefly indicate how meSsages are processed 
within the Host computers [15,32]. Messages travel through 
several layers of protocol ir the Host system. The first layer 


cf protocol handles the IMP, activating I/0 buffers, fielding 
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control messages, etc. The second layer interacts with the local 
processes and remote Hosts monitor, allocating buffer storaac, 
providinq process identification, formatting control information 
etc. Subsequent layers correspond to specific user oriented 
functions such as the standard network terminal, file 


transmission, etc. 


The ARPANET Host protocol utilizes the notion of connections 
over which messages are transmitted. A connecticn must first be 
established before communication over it may occur. ‘the Hosts at 
either end of the connection must keep full information about the 
use of the connection (which is obtained during its 
establishment) to hanéle flow centrol. This strateav anpnears to 


be close in spirit to telephone circuit switching, 


A few limitations to this strateqv are annarent [14]. An 
important concern is that it requires each Host to maintain 
resources in the foin of connection tables that can becone filled 
thus preventing anv further communication with that host. In 
particular, a Ssinale process can attempt to establish its maxir in 
limit of connections although it cannot , in general, maxe full 
use of them at one time. Entries in this connection table are 
permanently allocated and thus only a fixed number of connections 


can be established at any timc. 


A second limitation with the use of a connection table is 
that it can be vulnerabie tu error conditions an« Host status, 


particularly since both Hosts must generally aqree on its 
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contents for flow control. Finally the strategv requires the 
connection information to be used for termination which means 
that information which otherwise would be nominally discarded by 
Hosts with limited space, mist be retained merely to close the 
connection, These limitations , as well as others, may be 


obviated witn a message switched lHiost protocol [14]. 


The desire.i size of Host messages may have an important 
impact on the operatina systen as well as on the communication 
system, The original ARPANET™ design specification allowe: 
individual messages to be as laroe as 8192 bits, a desian choice 


based larqely on intuition. 


As the desian specification oricinally stated [29]: 
tae. a Packet a6 defined as “EhG. Inter=1)- tit sand 
fessage as the inter-Host unit. AS pachet wat lenot 
exceed 1;!24 bits in length. The IPs must break all 
longer messaces into multiple packets. Messanqes will 
be limite. to 3192 bits so as not to require excessive 

buffer snacc." 
Undoubtedly, this latter reference is interpretted as referrina 
to buffer space in tiie I!'Ps, but it could equally well apply to 
buffer snace in the Hosts. In particular, the araqunment defends 


why the size is not larqer but does not entertain the possibility 


that it ought to be kept smaller for any specific reason, 


If there 1s a convenient maximum Host mesSaqe size, it is 


probably a maxinun Sized page, which corresponds to 1K of 3v bit 
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( or possibly 48 bit) words. However, transmission of such larae 
messages (say 5ff,990 bits) to the IMP and from the IMP to the 
Host produces excessive delavs for short messaqes queued up 
behind then, and provides a prime reason for Hosts to prefer that 
these long messages be subdivided into shorter messaqes. Since 
no experience with network software was available during the 
initial design, it was intuitivelv concluded that a shorter 8192 
bit messaqe was Short enough. Interestindqly, we note that two 
hardware paths between each Host and its IMP, one for short 
messages and one for long messaaes, could remedy this problem at 


some extra cost in hardware and buffer storaqe. 


Since efficient transmission 1S possible with 1724 bit 
packets, it apnears in retrospect that the selection of the 
larger message Size mav be unnecessary. The Host overhead in 
network communication increases with the number c* messades, so 
there 1S some incentive for making all Host messaqes sufficiently 
large that a tvpical short transaction can occur ina single 
message. No evidence yet obtained by us indicates that 1°24 hit 
Host messages would inpose a limitation that 1S Siqnificant, but 
an increased deman: for pase transfers or the presence of higher 
bandwidth circuits could tiv the balance more stronagly in favor 


of a larger size. ‘The jurv Seems to be still out on this issue. 
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VI. APPLICATIONS OF MULTI-COMPUTER INTERACTION 


Network utilization involving the combined use of two or 
more computers ir a productive wav began during the initial 
experimentation with the ARPANET. It has provided experience in 
the development’ of techniques for performing distributed 
computation and allowed some simnle application areas to be 
identified. Some applications involving multiple computers have 
been discussed for many years, partly as a result of their 
inherent interest and ease of conceptualization. One important 
example is the access to snecialized data bases that are only 
available from a remote source. Several information banks have 
already been developed or are under development, and their 
expected usage is being projected unward. Another examnle is in 
the use of "future" computer communication networks for handling 
the distribution and delivery of mail and other’ transient 
information, However, these annlications are only beqinning to 
develop in any Siqnificant way. Much effort has already been 
devoted to the «:udy of tonics such as concurrency and parallel 
processing whicn mav result in faster program execution and 
otherwise make efficient usage of available resources. We expect 
that computer networking will enhance these efforts. For otner 
applications, the sensation of dealing with one svstem rather 
than two (or more) is overwhelmingly evident to the user and this 
pleasant feeling often generalizes to other multi-computer 


interactions as well. 
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Three areas in which applications have already occurred are 


briefly identified below: 


1. File Transfer 


The first application for combining two computer systems in 
the ARPANET in a non-trivial way involved the use of an XDS=-944 
computer at SRI and a PDPe-ll: computer at UTAH. SI, anticipating 
the delivery of a PDP-1), began to use the UTAH machine in the 


development of PDP=1) software. 


At first, a hiqher level lanauaqe was developec. Source 
coue was qenerated on the 94J, converted to object code ana 
executed on the 1%. Patches were made on the UTA! machine durina 
debugging and, periodically, an updated source and hinarv version 
would be generated at SRI and sent over the net. Subsequently, 


other higher level lanquaaes were similarly develonecd. 


A Simple protocol to handle file transfers was develo, ad for 
the THNEX operating svstem [10] and has proven’ useful for 
transferring new subsvstems and system revisions between TENEX 
Sites, In addition, it has been a useful initial sten to allow 
cooperating processes in two TNNEX systems to share ae sinyle 


file. 


In this protocol, the network appears as a device to which a 


file may be output or from which a file mav be input. The two 
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ends vf the transfer must coordinate by having one end execute 
the irput.and the other end -xcuuce the output. This simple file 
transfer protocol requires the intervention of the user to log 
into both ends, assign a file name for the destination, invoke 
the proper format, etc. Other experience in the transfer of 
files has been re.:orded by the Uniwarsity of California at Santa 


Barbara, as well as by 18M, by Control Lata and others [19,21]. 


ii.e Remote Job Service 


A Simple example cf a computer to computer interaction is 
provided by users who write, Czcgug, edit and store programs on an 
interactive time-sharing facility und run them on a_e separetc 
batch process.nq system. Wh.le time-sharing has created an 
interactive environment for programming and the development of 
programming techniques, batch processing systems (and small 
dedicated computers) have maintained a predominance for 
performing extensive computations. The availability of both 
kinds of service in a computer communications network provides a 
Single user with convenient access to the best features of both. 


See figure 3. 


While a user can become accustomed to using both services 
independently, he need noc be required to physically collect time 
Sharing output on tape or punched cards for submission to the 


batch system. The most convenient user option is for the 
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interactive machine to submit his job to the batch processing 
Machine under user initiated control. He can then specify the 
location for output to be stored or printed, revise the program 
in the time sharing system and resvrbmit it under fully 
interactive procedures from a single location, with no need to 


keep physical copies of files, program etc. 


For ovexr a year, the RAND Corporation had been using the 
ARPAKET for remote job service from an IBM 3609/65 at RAND to a 
368/91 at UCLA [26,27!1. This facility was only accessible to 
internal RAND asers wurtil recertly when it was replaced by a 
PDP=-19, which allowed 1etwork users to create and submit jobs for 


remste eervice clsoeewhere. 


Both <ne program ind its relevant files must be transferred 
to me 369/91 iLcfore a job can be run. They are typically 
shipped t.vtether as successS.ve "card images". The remote job 
Service program wili ailow the users to start or stop the 
execution of his prozram, cause the system output to be stored on 
a designated file, or be output on a device such as a local 
trinter. rhe user 1s also provided with options to check che 
status of the execution, receive confirmation and error messages 
that indicate its progress, and allow certain actions to be 


taken, 


These facilities are used by RAND researchers in the 


generation and processing of simulated weather data. Weather 
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modelling programs can be activated from a remote site, output 
from these programs can be temporarily stored or shipped to a 
renote site for preparation and display. This separation of the 
computation into components is particularly appropriate when one 
part may be devoted almost exclusively to extensive numerical or 
symbolic computation and another’ part to user related 


Manipulation of or preparation of the output data. 


A "complex" weather sinulation program requires many _ hours 
of computation on the 369/91 and thus is not well suited to rapid 
on-line activities such as the updating of a display. Rather, 
pre-computed weather data (from the models) is retrieved from 
369/91 disc packs (with operator assistance) and used by the 
PDP-19 for further -rocessing and display. The availability of a 
high speed parallel processing system such as the ILLIAC IV [33] 
may eventually ailow real-time weather experimentation without 


operator intervention, 


ili. Multi-Process Operation in Many machines 


The combined use of two or more computers allows additional 
processing capability over the use of a single system. One such 
example is provided by the McRoss system j{12] that coordinates 
the operaticn of two or more cooperating air traffic control 
simulation programs running in one or more TENEX sys’ 17s, Each 


Simulation program, called ROSS’ for Route Oriented Simulation 
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System [23], models the airspace of one air traffic control 
center in detail. To simulate the airspace of a Boston to New 
York flight, four simulation programs would be activated; ene for 
the Boston terminal area, one for the Boston enroute area, one 
for the New York enroute area, and one for the New York terminal 
area. The four Ross programs may be run similataneously in as 


many as four TENEX systems in the ARPANET. 


When a single machine is used to house all the components of 
a programming system, it has the disadvantage that comouting will 
stop if that machine crashes. When one piece of a multi-computer 
programming system becomes unavailable, the othex parts can learn 
to adjust to the change in configuration. A desireable objective 
is to provide enough backup information to eé..able the 
multi-computer programming system to be restarted in the event of 
a Single Host failure and to proceed froma recent point in 


Simulated time as if nothing had happened. 


Other applications involving multiple computers are certain 
to arise for which simple examples are more difficult to 
construct. For example, as Special areas of expertise develop, 
it is natural to expect that individual efforts by specialists 
also trained in the use of computers, will produce new and useful 
resources on different machines. These resources may represent 
state of the art or proprietary developments that cannot be 
conveniently transferred to other machines and must therefore be 


used at the site of their creation or where they currently exist. 
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An important application for distributed computation is thus 
likely to involve the coordination of separate research projects 
into combined’ efforts that utilize these specialized or 


proprietary and hence non-transferabl: resources. 


A second major application of distributed computation is 
likely to be the facilitation of interactive cooperation between 
people at different locations. Interactive cooperation may be 
regarded as an extension of normal voice communication to include 
the ability for several persons at different locations to 
“simultaneously" observe, communicate about, and manipulate both 
common data structures and prcgrams. Since the people are 
assumed not to be co-located, the programs which support the 
interactive cooperation (such as display protocol routine=) must 


also be distributed. 


A third major application for distributed computation is in 
providing for conveniently feasible demonstrations of prototype 
systems to be performed from differeat locations. This technique 
can allow new capabilities to be readily conveyed without the 


inconvenience of moving the observer to a home site for the 


demonstration. 
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VII. A DISTRIBUTED OPERATING SYSTEM 


Ah network in whicn basic differences exist between th- 
computers at each installation is said to be inhomogeneous. It 
is possible to develop a standard network protocol for an 
inhomogeneous system that allows usage of various pieces of the 
system to be coordinated in a uniform manner. However, this task 
is one of substantial complexity that will probably require 
changes in system architecture and program design techniques 
before Pe can be fully realized. Even if it were a 
Straightforward matter, it would not be generally useful to 
transfer portions of any one system to another, and standard 
operations that involve systems at a remote site must typicallv 


be performed at that remote site. 


A collection or Similar operating systems may also be 
Crganized into a virtual subnetwork of homogeneous computers that 
interact with each other ina uniform wavy. These systems are 
more easily orgarized anto a single distributed operating svstem 
with common file systems, address Space, naming conventions etc. 
In general, every type of interaction between two systems ina 
homogeneous network must be evaluated to determ>?ne what is to he 
transmitted and what is to be remotely evaluated. No single 
answer will suffice for all applications. As we noted above, it 
is not generally possible to apply both alternatives in an 
inhomogeneois network. We consider some of the properties a 


system like this ought to possess. 
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The user accesses the distributed homogeneous network by 
logging into a distributed system rather than into a specific 
computer in the net. An appropriate machine is selected for him 
and he logs in with the standard login sequence for his home 
computer, including password, account number and ocher 
infcrcmation as required. Upon completion of the login sequence, 
the computer initiates a bri2* exchange with the users’ home 
computer to notify it of the impending job which it then proceeds 
to service. The home computer may then request that the job be 
transferred, alter credit or accounting information, or merely 


note the event. 


Under conventional desiqn constraints, the combined 
operation of several Host computers will requira a separate job 
to be established in each macnine. In a distributed system, 
though, it is important to allow access to each system without 
the user logging into each syste individually. Furthermore, it 
is also desireable to permit certain transient activities to 
occur, which do not tie up valuable resources or otherwise 
interfere with users on the system. The system merely performs 
the transient activity and lcgs the transaction into a _ suitable 
file for accounting purposes. Once logqed into one of the Hosts, 
the user is able to acce:s and utilize any programs, files and 
most other facilities on other computers in the system as if they 


wer. all on one virtual machine. 


The availability of many resources in this system makes it 
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possible to achieve reliable operation when one or more resources 
are disabled. The user can be affected by failure in several 
ways, for example, his program or a pieve of it may be aborted by 
machine failure or he may lose part or all of his files. The 
user may also find the local file storage to be unusable while 
running his job. If Jocal storage is not initially available, he 
Can specifically designate another system to store his files. 
Alternatively, he can allow the local system to store files in 
other Hosts and expect them to be returned without his knowing 
the identity of the t>2mporary storage location. Obviously, a 
small amount of local stwrage is needed for this application. 
The distributed system thus not only makes’ resources more 
available, it can use them to provide increased reliability toa 


user. 


A system designed to ope. ate stand alone may not be as 
efficient is serving its network users as in serving its local 
users. Certain performance improvements are obtainable by 
Streamlining of critical portions of the system code, attention 
to organizational details and to carefully engineered 
improvements to scheduling, the file system etc. However, a 
major improvement in speed and efficiency may require structural 
overhaui of the system organization to allow for efficient 
process to process communication at high bandwidths, ard _ for 
efficient overall utilization of resources. In particular, the 


portion of the system devoted to protocol and message handling 


ay 


Report No. 2459 Bolt Beranek and Newman Inc. 


(byte manipulation) can consume a considerable amount of cpu time 
at high bandwidth. In a time sharing system, particularly, these 
functions, which appear to be communication functions, can be 
usefully separated from the “computation” functions. This 
decomposition also allows more human engineered network 
interfacing since, a local cpu failure can usually be reported by 
the communications portion of the system if it remains up. The 
delegation of all protocol functions to a separate processor that 
can directly deposit into and retrieve from process buffers makes 
it possible for the operating system to communicate over the 
network at speeds at least an order of magnitude faster than 
before. This increase in capability is achieved by performing 
the protocol operations on the fly in a separate processor and hy 
avoiding the unnecessary overhead in moving real-tire data around 


in memery. 


Techniques for computer to computer communication are still 
in their infancy and a greet deal of exploration and 
experimentation is occuring in this area. How should proqrams be 
written to run il:. a network environment and what debugging and 
control techniques are suitable for distributed computation? What 
operating system avcnitectures sare appropriate to computer 
communication? The efficient utilization of a distributed 
operating system involves the sensible decomposition of a task 
into components. This requires timely access to status 
information and the ability to use this informaiton wisely in the 


allocation of tasks to resources and in their scheduling. Just 
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as the management of communicution resources was central to the 
operation of a communication subnet, so will the management of 
computer resources be to the overall utilization of a computer 


network. 
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VIII. CONCLUSIONS 


A principal motive underlying computer network development 
is to provide a convenient and economic method for a wide variety 
of resources to be made available and to be. shared. Such a 
network provides more than an increased collection of hardware 
and software resources; it affords the capability for computers 
as well as individuals to interact in the exchange and processing 


of information, 


It is not uSually the case that a program written for one 
computer can be _ shipped to another computer and run there to 
completion, correctly. It may be possible in a number of cases 
where the machines are nominally identical, but it is usually the 
case that a program must be run on the machine for which it was 
written. It is thus desireable to stcive for compatibility 
between at least a subset of th2 system resources, including the 
use of machine independent higher level languages, the use of 
network wide standard protocols, c: the use of nominally 


identical systems. 


Tae development of communication subnets has been strongly 
influenced by the regulatory climate and the need for reliable 
and economic ways to achieve both remote terminal access and high 
bandwidth switched computer to computer communication. Message 
Switching has emerged as a strong contender for computer 10 
computer communications. It has been demonstrated to provide a 


nighivy reliable, error-free method of achieving interactive 
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switched commmunications. Although its technical feasibility has 
been firmly established, its practical utility is under 
evaluation, and under close scrutiny, it may prove to be a viable 


economic alternative to conventional circuit switching. 


It is important that a Communication system not preclude the 
possibility that separate or private data networks may be 
accessed through it in a Standard and convenient way. A_ digital 
message switched network has this property while an analog 
frequency based syst2m may not. Incompatible data networks are 
clearly undesireable if all resources are to be mutually 
accessible, If separate data networks are jointly planned before 
development, at least at the interconnection level, they may be 
connected at a later date and viewed together as a single network 


that evolved by by way of separate networks. 


The great diversity of resources in a computer network may 
initially hinder its growth. Users must familiarize themselves 
with many different systems often without the aid of substantive 
interaction with Systems personnel or clear and complete 
documentation. But the potential benefits of computer networks 
are sufficiently great that, over time, this obstacle will surely 
be surmounted and in the process may lead to Superior standards 


for system operation andi documentation. 


Computer networrs provide a unique mechanism for increased 
participation te.ween individuals. Participation in research and 


development using the distributed resources of a computer network 
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can lead to the close cooperation between individuals who might 
otherwise have little incentive to work together. This 
interaction can further cross fertilize the network commun:.ty and 


encourage even higher levels of achievement through technical 


cooperation. 


=48- 


a 4 


Report No. 2459 Bolt Beranek and Newmar. Inc. 


REFERENCES 


ve "Pinched Budgets Promote Growth of University Computer 
Networks," Communications of the ACM, Vol 15, No. 3, pp.286-297 

2. P.M. Walker and S.L. Mathison, "Regulatory Policy and 
Future Data Transmission Services," Chapter Nine of Computer 
Communication Networks, edited by Abramson and Kuo, Prentice 
Hall, 2973 

3. D.Farber, K.Larson, The Architecture of a Distributed Computer 
System An informal Description, University of California, 
Irvine, Technical Report NO. 11, Dept of information and 
Computer Science, 1976 

4. H. Baskin, B. Borgerson, R. Roberts, PRIME - An Architecture 
for Terminal Oriented Systems, AFIPS SJCC, 1972, po. 431-437. 

5. R. Davis, S. Zucker, C. Campbell, A Building block Approaczh to 
Multiprocessing, AFIPS SJCC, 1972, pp. 685-793. Burroughs, the 
interlock 

6. S. Crocker et ai, Function Oriented Protocols for the ARPA 
network, AFIPS SJCC, 1972, po. 271-279 

7. oH. Frank, R .E. Kahn, UL. Kleinrock, Computer Communication 
Network Design- Experience With Theory and Practice, AFIPS SJCC, 
1972, pp. 255-279) 

8. L. G. Roberts, B. Wessler, The ARPA network, In Comvouter 
Communication Networks, edited by Abramson and Kuo, Prentice 


Hall, 1973 


~49— 


Report No. 2459 Bolt Beranek and Newman Inc. 


9. M. Bee.e, N. Sullivan, Tymnet - A serendipitous evolution, 
Second ACM IEEE Conf. on Problems in the Optimization of Data 
Coommunication Systems, Palo Calif. 1971 

19. D.G.Bobrow, J.Burchfiel, D.Murphy, R.Tomilinson, TENEX - A 
paged Time-sharing System for the PDP-if, Comm. cf the ACM, Vol 
i5,; No. 3, March 1972, pp. 135=143 

11. F. 7. Corbato et al, An Introduction and Overview of the 
Multics System, AFIPS FICC 1965, pp. 185-196 

LUPE. ".Thomas, 0D. Henderson, McRoss - A Multi-Computer 
Programming System, AFIPS SJCC 1972, pp.281-293 

13. R.E.Kahn, W.R. Crowther, Flow Control in a Resource Sharing 
Computer Network, IFEE trans on Comm. Technology, Vol. COM-29, 
June 1972, pp. 539-546 

14. D.C.Walden, A system fcr Interprocess Communication in a 
Resource Sharing Computer Network, Comm. of the ACM, Vol 15. 
No. 4, April 1972, pp. 221-23) 

D5) The Host/Host Protocol for the ARPA Network, Network 
Information Center, Documerec No. :147, Stanford Research 
institute, Menlo Park, Cal. 

16. L.G.Roberts, &.Wessler, Computer Network Development to 
Achieve Resource Sharing, AFIPS SJCC, 1970, pp. 543-549 

1 P.Baran, Ss Boehm, P. Smith, On Distributed 
Communications, Series of Rand Reports, Rand Cornoratior Santa 
Monica Calif, 196€4 

18. F.Heart, R.E.Kahn, S. Ornstein, W.C wther, D. Walder., The 


Interface Message Processor for the ARPA Comuter Network, AFIPS 


Report No. 2459 Bolt Beranek and Newman Iac. 


SJCC, 1973, pp. 567 

19. A.H. We1s, Distributed Network Activity at IBM, IBM 
Research Report RC3392, June 1971 

28. H. Frank, I. Frisch, W. Chou, Topological Considerations in 
the Design of the ARPA Computer Network, AFIPS SJCC, 1979, pp. 
581-587 

21.W.J.Luther, Conceptual Rases of CYBERNET, Courant Symposium 
no. 3, 197%, Reprinted in Ccmputer Networks, Prentice Hall, 
Englewood Cliffs, pp. 111-146 

22. R. Van owlyke, H. Frank, Reliability of Computer 
Communication Networrs, Proc. of Fifth Conf. on ”pplications of 
Simulation, New York, Dec. 1971 

23. W. Cutheriand., T.Myer, D.Henderson, E.Thomas, Ross, A Route 
Oriented SimuZatior System, BYrOG. of Fatth Cont, on 
Applications o7# Simulaiior, New York, Dec. 1971 

24. T. Marill, L.G.Roberts. Toward a Cooperative Network of 
Time-Shared Computer: , FJCC 1966 

25. Project MAC Conference on Concurent Systems and Parallel 
Computation, Woods Hole, Mass., June i976 

26. EE. Harslem, unpublished memo, Rand “orporation, 1972 

27. R. Braden, unpublished memo, University of California at 
Los Angeles, 1972 

28. C.n. Fisher, Introduction to the DATRAN Switched Digital 
Netowrk, Proceedings of the International Conference on 
Communication, June 1971, pp. 23-1 to 23-3. 


29. Request for Quotation (Interface Message Processors for the 


-5le 


Report No. 2459 Bolt Beranek and Newman Inc. 


ARPA Computer Network), Defense Supply Service, Washington, D.C., 
July 1968 

38. R.A.Scantlebury, P.T. Wilkinson, The Design of a Switching 
System to Allow Remote access to computer Services by other 
Computers, Second ACM IEEE symposium on _ Problems in the 
Optimization of Data Communication Systems, Palo Alto, 
California, October 1971 

31. D.W. Davies, K.A. Bartlett, R.A. Scantlebury, and P.T. 
Wilkinson, A Digital Communication Network for Computers giving 
Rapid Response at Remote Terminals, ACM symposium on Operating 
System Principles, Gatlinburg, 1967 

32. Bolt Beranek and Newnan Report No. 1822, Specifications for 
the Interconnection of a Host and an IMP, Cambridge, Mass, 
$2138, October 1971 

33. W.J.Bouknight et al., The Illiac IV System, Proc. Of the 
I.E.E.E., Vol 69, No. 4, April 1972, pp. 369-388 


34, Datamation, April 1972, pp. 36-46 


ee 


cL/ 1/0 - AYOMLAN dWI-v_e 


SEN AYVH Nga YSANIL QNVY VION 


3SsvO M9 yaqqinog Hvin 4 HS 
NV IN397199W 


dW! IVWNINYSL 
AV 1dSIG 
SOIHd VHS 


WV49OUd dW! 
| 


(SGON MYOM LIN) 
dW! OL 


YjALNIYd 


DNS 


TWNIWYSL 


SAINI 
dN - 1VI0 


‘ dALAISL 
ate or \ W// —_ 


JaLNdWOd YSLNIdd JNIb——/ qWi HOLVE 
ONINVHS - SWIL Tindulo GNVESCIM — WNINY SL 
STIVNIWYSL 
LINODUID | 
qasval waALNdWOd 
is ONINVHS- SILL 
Sy 1dsid===/dWl \ 
Bi tee 
mae 
al 
Se ere SNOILOSNNOO LAN 4a LNdWNOO 
HOLVSE 


31g'ISSOd YSHLO 


mm 


YHOMLIN dW! JIdWIS “V 


Fa 


~~ 


